1. Technical Field of the Invention
This invention relates generally to system and methods for querying multimedia databases and, more particularly, to a system for generating digital images (stills and video) having various parameters recorded thereon, and system and methods for querying digital image (stills and video) archives by searching the recorded parameters, and for providing geographically visual user-requested directions.
2. Description of the Related Art
In general, owners of multimedia archives are continuously seeking to improve the management and search capabilities of their archives in order to make it easier for users of such archives to retrieve desired information. In particular, it would be useful to be able to query a multimedia archive comprised of digital images (e.g., photographs or video sequences) by specifying certain parameters which are recorded on the digital images such as names of geographic locations, altitude, longitude, time and/or date, and searching the contents of the archive to retrieve every digital image having the specified parameters. By way of example, in order to relive the experience of escaping a cold winter in the northeast USA, a person may retrieve every picture and/or video clip that was taken during their trip to Daytona Beach. This may be accomplished, for example, by generating a query which searches for digital stills and/or video (collectively, xe2x80x9cdigital imagesxe2x80x9d) having particular geographic data recorded thereon which falls within a specified geographic boundary associated with Daytona Beach.
There are various commercially available software application programs which can be utilized for creating, manipulating and redistributing digital photo albums (i.e., digital image archives). A few of these programs provide a user with limited image search capabilities. For instance, although several programs allow a user to search for particular images by date, the date refers to the date on which the image file was last modified and not the date on which the image was taken. Furthermore, there are no commercially available programs which allow a user to search for images by specifying a range of dates or range of time. Several of these programs allow a user to search images by color and/or texture, and can retrieve images that match a template image. Further, several of these programs allow a user to search images by file size, which has a limited value. In addition, several of these programs can sort images by date, file name, etc. A few have support to publish HTML web pages on the Internet and to e-mail them. In addition, there are no image album applications currently available which provide geographic search capability, e.g., none provide the capability to search-by parameters such as latitude, longitude, region and/or geographic location.
There is a need, therefore, in the industry to provide a digital image database query system which can search for digital images based on parameters such as geographic location, time and date. Moreover, a database query system having an image search program which uses such parameters, as well as additional parameters (e.g., camera parameters such as shutter speed, flash status, auto focus distance, light meter readings, focal length and aperture), would be even more useful for searching and retrieving images in a digital image archive having the recorded parameters. Indeed, such a system could automatically provide various details regarding the creation of the digital image, and would be of significant value to multimedia database search systems, as well as map and atlas makers, insurance agencies (e.g., real-estate, auto, and fire), hospitals, news agencies and crime investigating agencies.
Furthermore, a device that can automatically incorporate (i.e., record) user-specified parameters (such as latitude, longitude, time and data, as well as any other recordable parameters which may be unique to a particular image recording device such as a digital camera) onto a digital image would be a significantly valuable tool for facilitating the creation of digital image archives which can be utilized with image search systems based on such parameters. There is no device or system currently available that automatically records an adequate amount of parameters with each captured image or video such that the image or video can be sufficiently indexed so as to provide efficient image search and retrieval methods based on such parameters.
For instance, there are commercially available digital cameras which can record the time and date with each image, but recording only time and date with each captured image provides an insufficient number of parameters with which to index digital images for purposes of efficient database querying. There are commercially available film cameras which are capable of recording the time and date on each photographic image as well as additional parameters, but these devices do not record the additional camera parameters in an electronically accessible or searchable form. For example, U.S. Pat. No. 4,574,319 to M. Konishi entitled, xe2x80x9cElectronic Camera Having Non-Image Data Recorder,xe2x80x9d teaches the association of a magnetic recording area with each image for conventional film cameras. Camera parameters such as shutter speed, aperture value, frame number, location of image, and the name of the photographer for a given image can be stored in the associated magnetic recording area. The disadvantage with this camera is that, since the image is recorded on conventional film and the image related data is separately recorded in an electronic format, this configuration makes it difficult for these separate entities to be combined and be compatible with a database search and retrieval system. Indeed, Konishi does not elaborate on how such recorded data may be subsequently utilized.
There are commercially available digital cameras which allow other parameters to be recorded with the images: For example, U.S. Pat. No. 5,506,644 to Suzuki et al., entitled, xe2x80x9cCameraxe2x80x9d, teaches the incorporation of a Global Positioning System (GPS) receiver into a camera so as to record the location at which the image was taken. Suzuki, however, does not teach recording the plurality of parameters discussed above. Suzuki also teaches electronically or magnetically storing a plurality of place names and codes associated with the place names as a hierarchical table, which allows a user to search the table to select a particular place name to be recorded with a captured image. Suzuki does not teach searching and retrieving stored images using the recorded data (i.e., the places names).
Next, U.S. Pat. No. 5,247,356 to J. Ciampa, entitled: xe2x80x9cMethod and Apparatus for Mapping and Measuring Landxe2x80x9d, teaches the association of geographical location information with each image and stores the information in unused portions of an analog image, such as the vertical blanking interval, or in the border of a digital image. Ciampa also teaches a technique to capture images of overlapping portions of a strip of land and derive the elevation of natural geographic features found in the digital image and how to measure land by accurately scaling a video image of the land. Ciampa does not teach recording the plurality of parameters with each image as discussed above, nor does Ciampa teach using the recorded parameters for searching and retrieving the images from a database.
Further, U.S. Pat. No. 5,596,494 to S. Kuo, entitled: xe2x80x9cMethod and Apparatus for Acquiring Digital Mapsxe2x80x9d, discloses a method and apparatus for acquiring spatial imagery of terrestrial scenes and deriving the geophysical coordinates (latitude and longitude) of ground objects. This is accomplished with a computer-controlled camera which captures the images and records geophysical data via a GPS receiver. Various parameters such as roll, pitch, and drift angles of the camera frame are recorded with each image to correct for geometric distortion. The images along with the geophysical information is stored in a recording medium. Using this recorded data, it is possible to retrieve the latitude and longitude of any ground object. Likewise, Kuo does not teach recording the plurality of parameters discussed above (since it pertains soley to latitude and longitude) or an elaborate query system using the recorded parameters.
Next, U.S. Pat. 5,517,419 to Lanckton et al., entitled: xe2x80x9cAdvanced Terrain Mapping Systemxe2x80x9d, describes a system to identify and document terrain characteristics. The Lanckton system utilizes a GPS receiver to record geodite data and multiple cameras to record images. The images from a plurality of cameras provide a stereo characterization of the terrain. From these images, the system correlates image and position data for a given area of terrain and generates a database from which recorded position information relative to key terrain features may be retrieved. Again, Lanckton teaches neither recording the plurality of parameters discussed above nor an elaborate query system using the recorded parameters to retrieve images.
A digital image database having digital images with geographic data may also be utilized for providing visual directions. For instance, during in-route navigation, a person may want to specify an origin and a destination and be able to retrieve digital images (photos) of critical points such as exits, turn signs and landmarks along the specified route. In addition, a user may mark a specific route on a map in order to obtain all pictures taken along such route.
There are several commercially available map software applications which can provide the latitude and longitude of any point on the map. In addition, these map applications can show points of interest, such as hotels and restaurants, with geographic regions. With regard to road navigation, these systems can provide a turn-by-turn description for directions from one place to another. None of these systems, however, provide pictures of landmarks or critical points along a desired route so as to, e.g., assist a tourist in navigation.
The present invention is directed to system and methods for querying digital image archives containing digital photographs and/or videos (collectively, xe2x80x9cdigital imagesxe2x80x9d). The digital images are indexed in accordance with a plurality of recorded parameters including time, date and geographic location data (altitude and longitude), as well as image data such as lens focal length, auto focus distance, shutter speed, exposure duration, aperture setting, frame number, image quality, flash status and light meter readings, which are used for searching a database consisting of the digital images. These images are preferably generated by an image capturing system which is capable of measuring and recording a plurality of parameters with each captured digital image. The image retrieval system allows a querying user to search the image archive by formulating one or more of a plurality of query types which are based on the recorded parameters, and then retrieve and display those images having the specified parameters.
For instance, in one aspect of the present invention, a digital image retrieval system comprises:
an image database having a plurality of digital images stored therein, each of the plurality of digital images having at least one of a plurality of parameters associated therewith;
a geographic boundary database having geographic location data corresponding to each of a plurality of geographic locations;
means for inputting at least one of a plurality of image search queries into the system;
means for retrieving from the image database, in response to a corresponding one of the plurality of image search queries which specifies at least one of the plurality of parameters, digital images having the specified parameters;
means for accessing the geographic boundary database to retrieve geographic data of a geographic location in response to a corresponding one of the plurality of image search queries which specifies the geographic location, and for retrieving, from the image database, digital images having parameters which correspond to the geographic data of the specified geographic location; and
means for generating a map of a specified geographic location in response to a corresponding one of the plurality of image search queries which specifies the map of the specified geographic location and for determining geographic location data associated with one of a marked region or path on the map, and for retrieving, from the image database, digital images having parameters which correspond to the geographic location data of one of the marked region and the marked path; and
means for displaying the retrieved digital images.
These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings.